1. Field of the Invention
In general, the present invention relates to an image processing apparatus, an image processing method and a device manufacturing apparatus. More particularly, the present invention relates to an image processing apparatus which is capable of improving sensitivity deteriorations caused by the increasing number of pixels created on the display screen of the apparatus as pixels each having a small size and capable of increasing a speed when video is recording. The present invention also relates to an image processing method adopted by the image processing apparatus and relates to a manufacturing apparatus for manufacturing the image processing apparatus.
2. Description of the Related Art
Image taking apparatus such as a video camera include an apparatus which is capable of taking not only a moving image but also a static image. In an image taking apparatus provided with both a moving-image taking mode for taking a moving image and a static-image taking mode for taking a static image, when an image is taken in the static-image taking mode, signals are read out from all pixels provided on the display screen. When an image is taken in the moving-image taking mode, on the other hand, data is read out from predetermined pixels provided on the display screen in the so-called thinned-out read operation.
When an image is taken in the moving-image taking mode, the image processing apparatus adopts a frame-rate increasing method by which the thinned-out read operation is carried out, undesirably sacrificing pixel information displayed on the screen to a certain degree. Technologies underlying the method for increasing the frame rate are studied. The technologies include a technology underlying a method for increasing the speed at which pixel signals are read out and for carrying out the thinned-out read operation by devising the layout of wires and image taking devices. The technologies also include a technology underlying a method for optimally adjusting the sensitivity and the dynamic range.
Japanese Patent Laid-open No. 2006-319407 (hereinafter referred to as Patent Document 1) discloses an image taking apparatus which is switched from an all-pixel read mode to a thinned-out read mode and vice versa. Patent Document 1 shows the configuration of an image taking apparatus like one shown in a diagram of FIG. 1.
To put it in detail, the FIG. 1 is a circuit diagram showing R (Red) pixels, G (Green) pixels and B (Blue) pixels placed along the first to fourth rows and along the first and sixteenth columns. The figure also shows states of connections between the pixels and a first horizontal signal line group 15D and between the pixels and a second horizontal signal line group 15U. It is to be noted that, even though the following description explains only R (Red) pixels, G (Green) pixels and B (Blue) pixels placed along the first to fourth rows and along the first and sixteenth columns, the configurations of R (Red) pixels, G (Green) pixels and B (Blue) pixels placed along other rows and along other columns are the same as the pixels explained in the following description.
The first horizontal signal line group 15D is configured to include four horizontal signal lines, i.e., horizontal signal lines 15D1 to 15D4. By the same token, the second horizontal signal line group 15U is configured to include four horizontal signal lines, i.e., horizontal signal lines 15U1 to 15U4.
An R pixel 20 (1, 1) is placed at the intersection of the first row and the first column whereas a G pixel 20 (1, 2) is placed at the intersection of the first row and the second column. In this way, at the intersections of the first row and the third to sixteenth columns, R and G pixels are arranged alternately. By the same token, an R pixel 20 (3, 1) is placed at the intersection of the third row and the first column whereas a G pixel 20 (3, 2) is placed at the intersection of the third row and the second column. In this way, at the intersections of the third row and the third to sixteenth columns, R and G pixels are arranged alternately.
A G pixel 20 (2, 1) is placed at the intersection of the second row and the first column whereas a B pixel 20 (2, 2) is placed at the intersection of the second row and the second column. In this way, at the intersections of the second row and the third to sixteenth columns, G and B pixels are arranged alternately. By the same token, a G pixel 20 (4, 1) is placed at the intersection of the fourth row and the first column whereas a B pixel 20 (4, 2) is placed at the intersection of the fourth row and the second column. In this way, at the intersections of the fourth row and the third to sixteenth columns, G and B pixels are arranged alternately.
The first vertical signal line 17D (1) laid along the first column is connected to the R pixel 20 (1, 1) placed at the intersection of the first row and the first column as well as the R pixel 20 (3, 1) placed at the intersection of the third row and the first column. By the same token, any odd-numbered first vertical signal line 17D laid along an odd-numbered column is connected to the R pixel placed at the intersection of the first row and the odd-numbered column as well as the R pixel placed at the intersection of the third row and the odd-numbered column. For example, the first vertical signal line 17D (15) laid along the fifteenth column is connected to the R pixel 20 (1, 15) placed at the intersection of the first row and the fifteenth column as well as the R pixel 20 (3, 15) placed at the intersection of the third row and the fifteenth column.
The second vertical signal line 17U (1) laid along the first column is connected to the G pixel 20 (2, 1) placed at the intersection of the second row and the first column as well as the G pixel 20 (4, 1) placed at the intersection of the fourth row and the first column. By the same token, any odd-numbered second vertical signal line 17U laid along an odd-numbered column is connected to the G pixel placed at the intersection of the second row and the odd-numbered column as well as the G pixel placed at the intersection of the fourth row and the odd-numbered column. For example, the second vertical signal line 17U (15) laid along the fifteenth column is connected to the G pixel 20 (2, 15) placed at the intersection of the second row and the fifteenth column as well as the G pixel 20 (4, 15) placed at the intersection of the fourth row and the fifteenth column.
The first vertical signal line 17D (2) laid along the second column is connected to the B pixel 20 (2, 2) placed at the intersection of the second row and the second column as well as the B pixel 20 (4, 2) placed at the intersection of the fourth row and the second column. By the same token, any even-numbered first vertical signal line 17D laid along an even-numbered column is connected to the B pixel placed at the intersection of the second row and the even-numbered column as well as the B pixel placed at the intersection of the fourth row and the even-numbered column. For example, the first vertical signal line 17D (16) laid along the sixteenth column is connected to the B pixel 20 (2, 16) placed at the intersection of the second row and the sixteenth column as well as the B pixel 20 (4, 16) placed at the intersection of the fourth row and the sixteenth column.
The second vertical signal line 17U (2) laid along the second column is connected to the G pixel 20 (1, 2) placed at the intersection of the first row and the second column as well as the G pixel 20 (3, 2) placed at the intersection of the third row and the second column. By the same token, any even-numbered second vertical signal line 17U laid along an even-numbered column is connected to the G pixel placed at the intersection of the first row and the even-numbered column as well as the G pixel placed at the intersection of the third row and the even-numbered column. For example, the second vertical signal line 17U (16) laid along the sixteenth column is connected to the G pixel 20 (1, 16) placed at the intersection of the first row and the sixteenth column as well as the G pixel 20 (3, 16) placed at the intersection of the third row and the sixteenth column.
The first vertical signal lines 17D (1), 17D (2), 17D (15) and 17D (16) laid along the first, second, fifteenth and sixteenth columns respectively are connected to a horizontal signal line 15D1. The first vertical signal lines 17D (3), 17D (4), 17D (13) and 17D (14) laid along the third, fourth, thirteenth and fourteenth columns respectively are connected to a horizontal signal line 15D2. The first vertical signal lines 17D (5), 17D (6), 17D (11) and 17D (12) laid along the fifth, sixth, eleventh and twelfth columns respectively are connected to a horizontal signal line 15D3. The first vertical signal lines 17D (7), 17D (8), 17D (9) and 17D (10) laid along the seventh, eighth, ninth and tenth columns respectively are connected to a horizontal signal line 15D4.
It is to be noted that the first vertical signal lines 17D (1) to 17D (16) provided for the columns are connected to the horizontal signal lines 15D1 to 15D4 by a CDS (Correlated Double Sampling)/SH (Sample Hold) circuit and column select switches. However, the CDS/SH circuit and the column select switches are not shown in the diagram of FIG. 1.
The second vertical signal lines 17U (1), 17U (2), 17U (15) and 17U (16) laid along the first, second, fifteenth and sixteenth columns respectively are connected to a horizontal signal line 15U1. The second vertical signal lines 17U (3), 17U (4), 17U (13) and 17U (14) laid along the third, fourth, thirteenth and fourteenth columns respectively are connected to a horizontal signal line 15U2. The second vertical signal lines 17U (5), 17U (6), 17U (11) and 17U (12) laid along the fifth, sixth, eleventh and twelfth columns respectively are connected to a horizontal signal line 15U3. The second vertical signal lines 17U (7), 17U (8), 17U (9) and 17U (10) laid along the seventh, eighth, ninth and tenth columns respectively are connected to a horizontal signal line 15U4.
It is to be noted that the second vertical signal lines 17U (1) to 17U (16) provided for the columns are connected to the horizontal signal lines 15U1 to 15U4 by a CDS/SH circuit and column select switches. However, the CDS/SH circuit and the column select switches are not shown in the diagram of FIG. 1.
In each of portions above and below a pixel area of the Bayer array CMOS (Complementary Metal Oxide Semiconductor) image sensor adopting a column CDS method as is the case with one shown in the diagram of FIG. 1, a CDS/SH processing section, a column select switch sections and four horizontal signal lines are provided. If necessary, each of the column select portions is switched to select columns. By switching the column select portions, the read operation can be carried out in each of an all-pixel read mode, a 1/2 thinned-out read mode and 1/4 thinned-out read mode. That is to say, output processing can be carried out in a thinned-out read mode.
The configuration of another existing image taking apparatus is explained by referring to a diagram of FIG. 2. Much like the image sensor employed in the existing image taking apparatus shown in the diagram of FIG. 1, an image sensor 50 employed in the other existing image taking apparatus shown in the diagram of FIG. 2 is a CMOS image sensor of a column CDS Bayer array system. The image sensor 50 employed in the other existing image taking apparatus shown in the diagram of FIG. 2 is described in Japanese Patent Laid-open No. 2007-124137. The image sensor 50 shown in the diagram of FIG. 2 employs a pixel array section 51, left and right V decoders (or vertical scan circuits) 52-1 and 52-2, upper and lower H decoders (or horizontal scan circuits) 53-1 and 53-2, upper and lower CDS circuits 54-1 and 54-2, upper and lower horizontal select transistors 55-1 and 55-2, upper and lower horizontal signal lines 56-1 and 56-2, vertical signal lines 57-1 and 57-2 as well as vertical select lines 58-1 and 58-2.
The pixel array section 51 has a first pixel group composed of a plurality of pixels 59-1 laid out 2-dimensionally to form a pixel matrix and a second pixel group composed of a plurality of pixels 59-2 laid out 2-dimensionally to form a pixel matrix. Each of the pixels 59-1 and 59-2 employed in the pixel array section 51 has the configuration of a rectangular pixel employed in the image sensor shown in the diagram of FIG. 1 except that the pixels 59-1 and 59-2 are laid out in an inclined direction forming an angle of 45 degrees in conjunction with the vertical or horizontal direction. In this layout, pixels arranged on any specific pixel row are separated away from pixels arranged on a row adjacent to the specific pixel row by a distance equal to half the size of a pixel. By the same token, pixels arranged on any specific pixel column are separated away from pixels arranged on a column adjacent to the specific pixel column by a distance equal to half the size of a pixel. That is to say, the pixels 59-1 composing the first pixel group are separated away from the pixels 59-2 composing the second pixel group in both the horizontal and vertical directions by a distance equal to half the size of a pixel. Each of the pixels 59-1 and 59-2 includes a pixel circuit, and a color filter is set for each of the pixels 59-1 and 59-2.
A color matrix of a Bayer array is created on each of the first and second pixel group. The Bayer array has a typical 2×2 RGBG configuration. In the sequence RGBG, notation R denotes a red-color filter, notation G denotes a green-color filter and notation B denotes a blue-color filter. In the image sensor 50 shown in the diagram of FIG. 2, the Bayer array on the first pixel group is referred to as Bayer array 1 whereas the Bayer array on the second pixel group is referred to as Bayer array 2.
As shown in the diagram of FIG. 2, the 2×2 color matrix of the first pixel group overlaps the 2×2 color matrix of the second pixel group. The V decoder 52-1 selects pixels 59-1 of the first pixel group in the pixel array section 51 through the vertical select line 58-1 in row units. The selection process is carried out sequentially on a row-after-row basis starting from the bottom end, and pixel signals are read out simultaneously as a batch from each row. The CDS circuit 54-1 carries out a correlation double sampling process on the pixel signals, which are read out from the first pixel group of the pixel array section 51 in row units, in order to reduce reset noises.
In this case, an electric potential of a 0-level period following a reset period of the pixel signal is clamped to an electric potential determined in advance by making use of a clamp pulse. Then, a signal period of the pixel signal is sampled and held by making use of a sample-hold pulse S/H (Sample/Hold) in order to obtain a pixel signal having fewer reset noises. The H decoder 53-1 selects a pixel signal from pixel signals output by the CDS circuit 54-1 for 1 row sequentially, starting from the left end on a pixel-after-pixel basis. The transistor 55-1 for horizontal selection serves as a horizontal output circuit. In this case, a transistor 55-1 provided at a location selected by the H decoder 53-1 is put in a turned-on state, outputting the pixel signal sampled and held by the CDS circuit 54-1 to the horizontal signal line 56-1. The pixel signals sequentially output to the horizontal signal line 56-1 in this way form a first image signal HL1.
The first image signal HL1 obtained from the first pixel group is amplified by a later-stage amplifier not shown in the diagram of FIG. 2 and is then output to a recipient outside the image sensor. In addition, the V decoder 52-2 selects pixels 59-2 of the second pixel group of the pixel array section 51 through the vertical select line 58-2 in row units. The selection process is carried out sequentially on a row-after-row basis starting from the bottom end, and pixel signals are read out simultaneously as a batch from each row. The CDS circuit 54-2 carries out a correlation double sampling process on the pixel signals, which are read out from the second pixel group of the pixel array section 51 in row units, in order to reduce reset noises. The H decoder 53-2 selects a pixel signal from pixel signals output by the CDS circuit 54-2 for 1 row sequentially, starting from the left end on a pixel-after-pixel basis.
The transistor 55-2 for horizontal selection serves as a horizontal output circuit. In this case, a transistor 55-2 provided at a location selected by the H decoder 53-2 is put in a turned-on state, outputting the pixel signal sample-held by the CDS circuit 54-2 to the horizontal signal line 56-2. The pixel signals sequentially output to the horizontal signal line 56-2 in this way form a second image signal HL2. The second image signal HL2 obtained from the second pixel group is amplified by a later-stage amplifier not shown in the diagram of FIG. 2 and is then output to a recipient outside the image sensor.
Next, the pixel circuit of every pixel 59-1 and every pixel 59-2 is explained. FIG. 3 is a diagram showing the pixel circuits. Each of the pixel circuits is configured to employ a photodiode PD, a transfer transistor T1, a reset transistor T2, an amplification transistor T3 and a select transistor T4. The photodiode PD is an element provided with an opto-electrical conversion function and an electrical-charge accumulation function. The anode of the photodiode PD is connected to the ground. The photodiode PD carries out a conversion process to convert incident light into an electrical charge having a magnitude representing the quantity of the light and accumulates the electrical charge obtained as a result of the conversion process. The transfer transistor T1 is connected between the cathode of the photodiode PD and a floating diffusion section FD. The transfer transistor T1 transfers the electrical charge generated by the photodiode PD to the floating diffusion section FD with a timing determined by a transfer pulse TRS applied to the gate of the transfer transistor T1.
The reset transistor T2 is connected between a power supply and the floating diffusion section FD. The reset transistor T2 resets the electric potential of the floating diffusion section FD to the electric potential of the power supply with a timing determined by a reset pulse RST applied to the gate of the reset transistor T2. The floating diffusion section FD is also connected to the gate of the amplification transistor T3. The amplification transistor T3 is connected to the vertical signal lines 57-1 and 57-2 by the select transistor T4. When the select transistor T4 is put in a turned-on state on the basis of a pixel select signal SEL, the amplification transistor T3 asserts a voltage according to an electric potential, which is obtained as a result of amplifying the electric potential appearing on the floating diffusion section FD, on the vertical signal lines 57-1 and 57-2.
The image sensor 50 shown in the diagram of FIG. 2 has pixels of the first and second pixel groups separated from each other in both the horizontal and vertical directions by a distance equal to half the size of a pixel. The image sensor 50 executes control so that the electrical-charge accumulation time for the first pixel group is different from the electrical-charge accumulation time for the second pixel group. Then, image signals obtained from the first and second pixel groups are synthesized to produce an output image signal. As a result, it is possible to easily obtain sensitivity and a dynamic range which are optimum for the image.